Numerical Methods in Markov Chain Modeling

This paper describes and compares several methods for computing stationary probability distributions of Markov chains. The main linear algebra problem consists of computing an eigenvector of a sparse, non-symmetric, matrix associated with a known eigenvalue. It can also be cast as a problem of solving a homogeneous, singular linear system. We present several methods based on combinations of Krylov subspace techniques, single vector power iteration/relaxation procedures and acceleration techniques. We compare the performance of these methods on some realistic problems.